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approaches to delivering academic quality and quality assurance in teaching and learning make effective, 
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I. INTRODUCTION 

The United States is in a bind. On the one hand, we need millions of additional citizens with at least one 
year of successful post-secondary experience to adapt to the knowledge economy. The Gates and Lumina 
Foundations, as well as President Obama have championed this goal in different ways. On the other hand, 
we have a post-secondary system that is trapped between rising costs, falling appropriations, and stagnant 
effectiveness, seemingly unable to respond effectively to this challenge. This paper analyzes several 
aspects of this problem, describes changes in the society that create the basis for solutions, and offers 
several examples from Kaplan University of emerging practices that suggest what good practice might 
look like in a world where quality-assured mass higher education is the norm. 


In The Long Tail [1], Chris Anderson identified and evaluated the world-wide web’s capacity to store 
unlimited amounts of data, only for the cost of entering it. The term “long tail” implies the explosion in 
widespread availability of items that have not, historically, been available due to insufficient demand and 
limited inventory space. Although his main example was the recording industry, the long tail message has 
major implications for other sectors, including higher education. The long tail signals the end of 
information scarcity and changes the rules that govern the relationships among individuals and 
institutions. In higher education, the end of information scarcity is a core driver in the generation of a new 
ecology of learning that makes the scale of operations for supporting high quality mass higher education 
achievable for the first time. When coupled with the consistency and flexibility of web-based 
programming, abundant information heralds a new age of learning opportunities. In this new age, 
technology enables us to serve millions of additional people with consistent and high quality. 


Several recent books, including DIY U [2] and Harnessing America’s Wasted Talent [3], explore the 
impact of this new ecology of learning on the traditional practices and assumptions of higher education. 
Technology’s value in the new ecology of learning has expanded beyond providing more access to 
rewriting the rules governing the traditional value proposition that is higher education. This “rewrite” 
includes assuring academic quality more consistently and reliably, with more personalization and 
customization, better student diagnostics, and better learning support in mass higher education. 


This is indeed a new frontier of learning. And it makes possible new approaches to delivering academic 
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quality and quality assurance in teaching and learning. Traditionally, the hierarchy of quality in higher 
education has been defined by accreditation, institutional reputation, and the quality of incoming students. 
The classroom and the curriculum have largely remained the private sanctum of the faculty member, as 
has the evaluation of learning. From an institutional perspective, assessing the quality of what is actually 
learned by each student at the course level is effectively impossible because the content as well as the 
standards and the evaluation of learning are interpreted by each faculty member individually, rendering 
comparison meaningless. 

A. Guiding Assumptions 

Historically, individual faculty members determined how well you did as a student in the courses they 
taught. Assessment was a faculty prerogative free from any serious oversight. Today, if we were 
educating the “sons of Harvard,” we could probably still get away with it. We know that strong prior 
educational performance is the single best indicator of future educational performance in college. So, until 
the fathers and mothers of those students can’t afford to pay the steadily-escalating bills associated with 
that model, technologically enabled efficiency and effectiveness will take a back seat to prestige. 
Furthermore, the inputs at a place like Harvard—students, faculty, other resources—coupled with its 
reputation are quality assurance enough for most people. There are, however, two further problems with 
this approach. First, the “quality via inputs” model won’t scale to meet the requirements of mass higher 
education. Second, for the rest of us, including the burgeoning proprietary sector, making sure that all 
learners, including multiple risk factor learners, are getting what they paid for at a price they can afford is 
the accountability target of the 21 st century. 


When we look at the alternative pathways to mass, high quality postsecondary education, it is clear that 
technology encourages and supports learning operations at a scale and scope that were unthinkable 10 
years ago. New media is changing the way learners behave, and learning is accessible to many millions 
more people than ever before. With the advent of mass higher education, the “learning platform” (or its 
future derivatives) will become the organizing architecture of college, not the campus. And, 
correspondingly, learning networks (or their future derivatives) will be the defining process for much of 
learning, not faculty governance. Although campuses and governance will remain in place, they will not 
be sufficient to assure external expectations for academic quality, effectiveness, efficiency, and success. 


Higher education is in the early stages of a seismic shift away from curriculum and teaching and towards 
learning outcomes, learning support, and assessment as quality differentiators. For-profit education and 
other private-sector interests will play a significant role in defining and developing the potential in this 
migration. The sector will lead the way not because it is more virtuous, per se, but because it is not tied 
down politically to state funding traditions or organizationally to restrictive traditional academic 
processes and practices. My experience in both sectors tells me that the sector will drive innovation, 
spawning change and improvement, because it is able and motivated to do so, responding to the new 
markets, new learners, and new opportunities generated by technology. 


These factors —the shift towards learning outcomes and assessment, an increasing interest and capacity 
for effectiveness and efficiency in learning, the new understanding of educational quality and 
technology’s role in supporting it at scale, and the sector’s ability to drive innovation and change— 
suggest the implications of this new ecology of learning. 


B. Three Potent Possibilities 

Technology offers institutions many potent possibilities including the following abilities to focus on: 
the consistency and the quality of what is learned using learning outcomes and rubrics at both the course 
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and program levels, 

the effectiveness and interplay of learner support, diagnostics, and teaching to build in and secure 
academic quality for all learners, including historically marginalized populations, and 
scaling services and quality assurance to previously unimaginable numbers of learners. 

These three points are important because breaking the barriers to scale with quality is essential if 
postsecondary education is going to be available to the mass of America’s population. And they illustrate 
that technology is a key quality differentiator and well as essential enabler in achieving mass higher 
education. 

II. SCARCITY: THE OBSTACLE TO ACHIEVING MASS HIGHER 

EDUCATION 

Higher education’s greatest strength, as it has been designed, built, and developed, is a great weakness 
when it comes to addressing many of the new challenges facing America today. Although we have 
redefined and extended higher education opportunity dramatically over the last 60 years, our system of 
higher education is still organized around the principle of scarcity. The scarcity principle assumes two 
things: 


First, that the resources needed to provide an education must be collected in one place, a campus, because 
there is an insufficient supply of those resources in the general community. 

And second, that there is a scarcity of jobs in the larger society requiring more than a high school 
education, creating the need for a meritocracy to decide who would compete for them. 

More generally, the principle of scarcity says that, for an institution to be valuable to the community 
around it, it must offer a service that community members can’t get more cheaply or with higher quality 
somewhere else. Scarcity, based on an inability to duplicate the resource, controls the market. 


Scarcity has been a reality in the world of information, teaching and learning. Organizing scarce 
community resources to create valuable institutions—colleges and universities—has served us well, 
creating educational opportunity while building research centers and professional education programs of 
unparalleled quality and depth. In fact, there was no other choice, no other way to “do” higher education. 
Colleges are built around the faculty and the curriculum. They organize facilities, libraries, and 
laboratories (not to mention gorgeous dorms and playing fields) to host, support, and entertain the 
students who come to learn. They compete for students and market to students based on the quality of the 
living and learning experience offered by the campus. And faculty members are paid to teach, to assist in 
the governance, and to do research. Finally, as we know all too well, there are funding and physical space 
limits to the number of students who can attend a given college at any one time. 


As a scarce commodity, access to college has traditionally favored people who had time for college, could 
afford college, and could learn effectively in colleges the way they were structured. Scarcity created a 
standard for access that favored some learners over others. Those who were less favored either learned 
differently, had to work, or were constrained by other social and economic forces. For years, incentives 
such as student financial aid, remedial education, and enrollment management services have been 
employed to level the playing field and support access for those who had been marginalized. There was, 
however, no alternative to the capacity limits inherent in the traditional model and its campuses other than 
building more of them. 


As successful as higher education has been in so many ways, this natural scarcity has generated perverse 
parallel effects. It has become part of our cultural mythology that says to those who cannot attend college 
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as it is positioned, “There is a reason for this scarcity of opportunity. Opportunity is not for you. It’s for 
the ‘smart’ people, those who can benefit from college.” The “scarcity’ model leaves out otherwise 
capable people who, for whatever set of reasons, find themselves on the margins of social, economic, and 
educational activity, struggling to get a seat at the American table of opportunity. That works for 
institutions that are filled with students who fit their mold. And it works in societies where there are a 
limited number of “knowledge-based’ jobs. But it doesn’t work for the capable learners who are left on 
the outside or for the larger society with significant vacancy rates in many higher skill jobs. 

Now, however, all this has been flipped upside down. Scarcity has become a trap. Stalled by diminishing 
returns and soaring costs and surrounded by information abundance, traditionally organized institutions 
are no longer the only source of solutions. Concurrent with the emergence of abundant content, there is a 
significant and growing need not only for more higher education opportunity, but for more attainment and 
completion as well. 

III. ABUNDANCE: DRIVING A NEW ECOLOGY OF LEARNING 

In this section, 1 will describe three examples that underscore the impact of abundance on the changing 
education and learning landscape. The examples show how abundance overcomes the barriers established 
by scarcity as we have experienced them, creating a new ecology of learning to improve academic quality 
while serving many more people. 


In the age of abundance, we have the ability as well as the need to make mass higher education 
achievable. Today we are faced with a significant challenge to improve and recognize the learning that 
millions of Americans do, strengthen our society, and create a more educated workforce to fill the jobs of 
the future. Conditions in society, beyond our campuses, are aligning to create an abundance of learning 
opportunities, using new tools and knowledge to change the rules, reinterpret the higher education world, 
and create higher education opportunities for previously under-served populations. 


A. A Future of Unlimited Content 

When the Community College of Vermont, Empire State College and several other non-campus colleges 
were founded in the early 1970’s, they represented a philosophical statement and commitment to take 
learning to learners in off-campus settings and to be responsive to the needs of learners. But in the 
“scarcity” environment of the 1970’s and 1980’s, these innovative institutions were still political and 
educational alternatives to the campus model. They employed new, extended versions of the dominant 
teaching-learning approach. 


The Open Education Resource (OER) movement, iTunes University, and the Global OpenCourseWare 
consortium symbolize something very different: an era when excellent, organized curriculum is at the 
fingertips of every potential learner. These free resources obliterate the traditional boundaries that defined 
the campus and its schedule, spawning multiple possibilities for providing and supporting organized 
learning. A content-rich world, accessible and diverse, is developing more rapidly every day. Technology 
puts curricular scarcity out of business, transforming that part of the higher education value proposition. 
Access to superb curricular content is no longer controlled by the faculty. 


Now, thanks to the “long tail,” you can get the course you want, when you want it, for your own personal 
learning and pleasure or to enhance your formal learning. It is at your fingertips, customized with search 
engines for the need or interest that you have. We are entering an age when more people, using fewer 
resources per person, will have access to a previously unimaginable variety of information resources. 
And, just as with iTunes AND MUSIC, previously excluded people will be able to get the exact 
information they want, when they want it. 
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People who want a higher education, including those who have not had access to a higher education, will 
have new choices. Policy-makers looking for a way to bring millions of Americans from the margin of 
economic opportunity to the mainstream of social, economic, and civic life have new choices as well. We 
need an additional, entirely different set of educational approaches if we are going to successfully harness 
the potential that the emerging “Long Tail” of course content brings to the critical national need for a 
larger, better-educated workforce to meet the demands of the emerging “knowledge” economy. 


Here is just one conception of the opportunity for qualitative re-design of higher education that lies before 
us. Mark Hopkins, the renowned philosopher-educator, introduced the concept of “sitting on a log” with a 
student, discussing knowledge. This extension of the Socratic dialogue, long held as one of the finest 
ways for teaching and learning to happen, is now possible directly and indirectly, anytime, anywhere with 
a mentor, a teacher, or fellow learners, formally or informally via the web. The growing availability of 
high quality curricular content can provide learners with online, blended, and face-to-face opportunities 
for high quality teaching and learning that have not previously existed. We have returned to one of the 
taproots of western culture, an historic foundation for teaching and learning, the one-to-one or small 
group conversation, only to discover a previously unimaginable future. 


Abundance of content changes forever the relationship between colleges and the learners who seek the 
education and credentials they offer. As surely as spring follows winter, people will become increasingly 
comfortable with choices that are available, accessible, and cheap when it comes to pursuing learning 
outside of colleges’ traditional instructional format. The world, and the relationship between the learner 
and the university, is changing, driven by web 2.0. 


B. A Future of Unlimited Access 

Technology also turns the access barrier upside down. Physical scarcity has driven the educational 
equation since the first itinerant monk walked rural Europe. The resources available to educate people, 
compared to the need, have always been scarce—too few books, too few faculty members, too few 
campuses, too few libraries and laboratories. Campuses reflect the same scarcity through their physical 
limitations and their funding formulas. 


Today, we face an unprecedented situation. College campuses, organized around the principle of physical 
scarcity, were built on the assumption that the campus was necessary to the overall endeavor of learning. 
They now, however, exist in a world where there are few physical barriers to learning. Even laboratory 
courses can be offered via the web. The only actual remaining barriers to access are capacity to do the 
work and ability to pay. Libraries are still important, but learners have access to previously unimaginable 
intellectual resources to support their learning from a wide and expanding universe of sources. 

Since Allen Tough identified personal learning activities as “the adult’s learning projects,” we have 
known that most adults learn continually. Tough’s research quantified the personal, self-directed learning 
that everyone does. The average adult spends about 15 hours a week (800 hours annually) engaged in 
personal learning via discrete learning projects. While the number of projects varies, most people engage 
in them and they range from 2 to 20 with a median of 8 projects annually [4]. 


When you couple this instinct to learn, and the learning projects themselves, with the resources available 
via the web, the opportunity for people to learn becomes turbo-charged. Is it any surprise that, in this 
environment, an increasing number of learners, both formal and informal, are ahead of colleges and their 
faculties in their learning patterns? 
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These learners understand and are using the technology and the content available to meet their learning 
needs. For example, in 2009 almost 50% of learners downloading courseware from MIT’s Open 
Courseware site were self-described self-learners. This was a big surprise to the organizers who had 
anticipated that the users would be other college students and faculty. And many of these “self-described 
self-learners” did not hold a Bachelors degree. For them, this is more than a recreational activity. 

This is happening at a time when the demand as well as the need for higher education is soaring. We need 
to bring millions of additional capable learners from the margin of our society to the mainstream of 
economic opportunity to fill the new jobs being created. Abundant access is our friend. 


C. A Future with Abundant High-Skilled Jobs 

The workplace demand for people educated with at least an associate’s degree has reversed itself. For the 
first time in our history, we have more jobs requiring higher order learning and education than we have 
people to fill them. This remained true even during the downturn in 2008-2009 when jobs requiring 
specific skills (e.g. nursing) remained unfilled even as the unemployment rate was rising. In fact, a 
survey of the 175 fastest growing occupations in 2008 revealed that most of them required at least an 
associate’s degree, with many preferring the bachelor’s or higher [5]. This new demand has created a new 
social imperative and a new marketplace for higher education. We can no longer assume a scarcity of 
need for educated people. We must organize for abundance, generating every trained and educated person 
we can get. 

Let’s review some salient facts. 

Growth in the number of working Americans has slowed as population growth has slowed. 

As our population ages, the percentage of people in the work force is declining. 

Year-olds have a lower high school graduation rate than that of older Americans. 

By 2014, 78% of all jobs will require some postsecondary education. 

Today, only 35% of Americans have at least an Associate’s Degree. 

There will be a skilled labor shortage of 7 million jobs by 2010 and 21 million jobs by 2020 [6], 


The entire higher education system has been built around the assumption that there were fewer jobs that 
required a college education than there were qualified people to fill them. Indeed, the very concept of 
“meritocracy,” the Jeffersonian principle that underlies so much of our American promise, assumes that 
getting the “good” jobs should be based on merit. That makes sense. But what happens if there are more 
“good” jobs than there are people to fill them? The answer, from a domestic policy perspective, is 
“nothing good” for either the society or the people. 


As we already know, however, when people learn something, they want to get formal recognition for their 
learning. This instinct, to be recognized and validated for what you know and can do, is forming a rising 
tide of aspiration in millions of learners that laps against the walls of our colleges and universities. In this 
emerging marketplace, our educational system needs to be creating and validating merit. We need to 
figure out how to recognize capacity in all its forms, not screen it out. Pat Callan, President of The 
National Center for Public Policy and Higher Education, addressed the issues of cost and diminishing 
returns as he explained the Center’s 2008 report on college affordability, 

“When we come out of the recession, we’re really going to be in jeopardy, because the educational gap 
between our work force and the rest of the world will make it very hard to be competitive. Already, we 
are one of the few countries where 25- to 34-year-olds are less educated than older workers.” [7] 

If we can harness the potential of these three forces - abundant content, access, and jobs - in new 
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educational forms and programs, we can avoid the jeopardy that Callan foresees. And, in the process, we 
can use technology as a quality differentiator in mass higher education. 


D. Obstacles to Adoption 

Scarcity of resources, the scarcity assumption, currently describes a choice, not a necessity. Colleges’ and 
universities’ models of operation come from a different time, a time when you had to ride your horse 
across town to see if your friend was home to chat. The ecology, economics, and delivery processes that 
come with abundance run counter to those of the traditional model. 


Two additional factors complicate our adoption of the necessary changes in how we think about higher 
education. First, we, as individuals and institutionally, can be slow to recognize, grasp, and accept large- 
scale changes that come at us like this. The significance of the end of scarcity, and the consequences that 
follow on its heels, are either denied or misunderstood by many both inside and outside of education. We 
like our traditions because they are familiar, a habitual way of understanding the world around us. They 
give us comfort and organize our world. 


Second, as a consequence, the technology driving abundant information is under-utilized in higher 
education while some of its implications are ignored because they challenge many traditions. As things 
currently stand, technology is, more often than not, integrated with the curriculum and the organization of 
the university as it has historically been organized. So podcasts of lectures are available for the learners. 
Or they are used to maximize convenience, getting students quicker up-to-date information about events 
on campus. 

But, more fundamental questions might include: 

• How might an institution integrate Google, the OpenCourseWare Consortium, or iTunes 
University into the base of its approach to curriculum, teaching and learning? 

• Can we guarantee the reliability and consistency of the learning we recognize? 

• Or will the evaluation of learning remain the personal interpretation of an individual faculty 
member? 


Most colleges have held the information revolution and the technology that supports it at arm’s length. 
They have asked the question: how does it help us do what we already do, only better? And, as a result, 
the major breakthroughs to date have not been in new learning design and assessment, or portability. The 
breakthroughs have been with access to education, getting material to people who couldn’t get it before. 
Although this is important, it is a gross under-utilization of technology’s educational potential. 


As a practical matter, if we are going to double our graduation numbers as the President and other 
educational and foundation leaders want us to do, we will need to reach beyond the traditional college 
model to succeed. Using the existing higher education model, we would need to dramatically increase the 
number of faculty teaching in our universities. 1 see no political calculus which would successfully gamer 
the funds necessary to train and employ them. And the demand for top people in the science, technology, 
engineering, math and language areas outside of higher education will also defeat this goal. People with 
the necessary qualifications can make significantly more money working outside the academy. I see no 
analysis that suggests this reality will change to favor traditional higher education. So, even if we could 
afford to increase our traditional capacity by 50-75%, there will not be sufficient qualified workforce 
available in the traditional format to fill the needed positions. 

The requirement for our existing colleges and universities is clear. We need our traditional system of 
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higher education to continue as it is, with continuing improvement, effectiveness, and efficiency to hold 
the center of the spectrum and sustain America’s educational greatness in its traditional format. Our goal 
should be to achieve a higher student success rate with the spaces and facilities we have. 


Current trends suggest, however, that as tuition is increased to offset budget decreases from other sources, 
the amount of money actually going to instruction and student support will decline. The law of 
diminishing returns suggests that future improvements in the success rates will be marginal in relation to 
the problem that needs to be solved. So, if we do not innovate and improve, millions of learners will 
remain on the outside, looking in. It is truly ironic. But as successful as our traditional higher education 
model has become, what got us where we are won’t get us where we need to go. 


We need a dramatic new vision for the future with new practices and new types of organizations and 
institutions to achieve the increased academic success we need to survive. This reformulation of how we 
offer higher education is now possible. Web 2.0 has created an ecology of abundance, making the 
availability of content, access, and a trained workforce universal. 

IV. THREE EXAMPLES FROM KAPLAN HIGHER EDUCATION 

The for-profit sector has the potential to be a center of innovation as we adapt to the new ecology of 
learning. While a culture of preservation will limit some ventures, others in the sector will continue to 
innovate, testing new approaches and then doing more of what works best; letting metrics and learner 
performance drive education decisions. And the essence of the innovations will go beyond “delivery” 
modalities to a rethinking of curriculum and student diagnostics and support as well as a redefinition of 
quality. 


As the following three examples suggest, quality will be determined by a measured and evaluated process 
of learning, not by the reputation of faculty or an institution’s history. Improved academic quality and 
quality assurance for all learners are the emerging core requirements for higher education. These 
examples describe areas in which Kaplan is innovating to clarify its academic standards and better 
measure academic attainment. In each case, the innovation would have been impossible to develop and 
take to scale for large numbers of learners and faculty without information technology. 


A willingness to think differently is one thing. Kaplan did that with the “Kaplan Commitment,” an 
innovative financing program that allows all incoming learners to take their first degree-oriented course 
with an option to leave within the first 5 weeks with no financial risk or academic penalty. Thus, if they 
change their mind, or if we think that they are not exhibiting sufficient academic performance, they can 
leave without penalty. 


It is another thing entirely, however, to redesign processes, systems, and approaches in a continuous 
improvement process that moves the University and its learners steadily forward towards higher academic 
quality, improved persistence and graduation rates, and better employment outcomes. In these academic 
activities, the concept of academic quality includes appropriate learning that has been achieved in other 
formal settings as well as informal knowledge, skills and abilities acquired through OER resources or 
broader life experience. 


Academic quality also extends the traditional conception of “being smart” to a broader understanding of 
capacity that includes being ready and capable of contributing to society civically, socially, and 
economically; or, more succinctly, being ready to work. This is a critical distinction. How many times in 
higher education have we graduated successful students only for them and employers to learn that they 
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are not ready to be successful in the workplace or be productive as a team member in the workplace on 
day one? 


This broader understanding of what it means to be well-educated will be underpinned by learning 
assessments that are valid, consistent, reliable, and tied to the knowledge skills and abilities that 
citizenship and the global economy require. And these assessments will be evaluated themselves for 
consistency by independent third party reviewers, on a continuing basis. 


A. Course Level Assessments (CLA) 

In an increasingly mobile society, with burgeoning opportunities to learn throughout life, the accuracy, 
clarity, and transparency of learning assessment at the course level is essential. Academic outcomes that 
are consistently and rigorously assessed by the institution and validated by third parties will be seen to 
have integrity in the 21 st century, earning the respect and confidence of employers and students alike. 


Course level assessment (CLA) is one of Kaplan’s answers to this emerging need. CLA measures student 
learning and informs our continuous academic improvement process with processes that have been 
developed to indentify and assess learning outcomes. CLA measures student mastery of stated course 
level learning outcomes in an objective way. It is criterion-references, not norm referenced. The scores 
obtained measure the student’s current skill mastery and knowledge described by the outcomes. CLAs 
support program-level outcomes while providing the framework for assessing specific learning objectives 
and activities within a course. Outcomes also share the following characteristics: 


• Each describes one primary area of knowledge or skill. 

• Each reflects specific behaviors underlying the knowledge or skills for which students should be 
able to demonstrate mastery by the end of the course. 

• Each is written in a style that reflects the appropriate level of complexity of the underlying 
cognitive tasks required for given levels of mastery. 


Tracking student learning outcomes at the course level allows us to gauge both the effectiveness and the 
career relevance of our instruction and our curriculum—and to engage in a continuous improvement 
process. 


The learning outcomes are supported by rubrics at the course level. For each course, faculty members 
assess student success in all of the course’s outcomes using standardized rubrics. Rubrics are developed 
for each outcome based on specific criteria, identifying student progress towards Mastery. Scores on 
outcomes are then analyzed to determine if students are gaining the desired mastery. And as students 
progress through their programs of study, their progress on achieving these outcomes is monitored. Figure 
1 portrays some of the early data collected via this method [8]. 
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Results 


Student Learning Over Time at Kaplan University 



■ 5*Mastery ■ 4-Proflclent ■ 3-Practiced ■ 2-Emegent ■ 1-lntroductory ■ 0-No Progress 


•Q12009 -77% 
demonstrate practice or 
better 

•Ql, 2010 - 83% 
demonstrate practice or 
better 

•Q4, 2010 - 87% 
demonstrate practice or 
better 


Figure 1. Consistent gains since implementation of course level assessment 


This approach allows us to achieve a high degree of consistency and academic quality in multiple ways. 
They include: 

• Consistent and transparent standards and learning expectations for participating students across 
every section of the every course. 

• Equally consistent and transparent expectations for the student-facing experience as well. 

• A consistent curricular and outcome structure within which faculty teaching can flex and adapt 
given learner needs and faculty strengths. 


So, the CLA gauges student progress on mastering the general education literacies and discipline specific 
course outcomes throughout the learner’s degree program. We evaluate on a 0-5 scale with 0=no 
progress, 3= “practiced,” and 5=mastery. The objective is that each learner will reach mastery of 
discipline course outcomes by the end of the course and mastery of general education outcomes by degree 
completion. 


The rubric structure allows us to look at the “profile of learning” within a section or across all sections of 
a particular course to identify anomalies and success rates as well as levels of learning [9]. Technology is 
the key differentiator in being able to gather and analyze this kind of data. 

• First, technology allows us to scale this research to all of our learners, ultimately collecting 
information on hundreds of thousands of student/courses per year. 

• Second, technology allows us to collect consistent information across every section, something 
that would be impossible in a traditional institution’s classrooms. 

• And third, we know that we have clear control over the means and structure of learning 
assessment, leading to a high degree of consistency in that process as well. 


B. Using a Curricular Matrix to Deepen Learning and Improve Curriculum 
Quality 

Our use of technology also gives us the capacity to embed certain learning outcomes across the 
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curriculum in a matrixed approach. So, in a single course experience, we can evaluate the student’s 
mastery of the substance of the course. We can also, however, evaluate other knowledge development 
such as writing, team-work, or critical thinking as well. 

For example, General Education at Kaplan University is taught through a core curriculum of six courses 
with other outcomes distributed throughout the undergraduate curriculum. The overall program goal is for 
the student to be literate and knowledgeable in the following nine areas. 

Arts and Humanities 
Communication 
Critical Thinking 
Ethics 

Mathematics 

Research and Information 
Science 


The distribution of outcomes allows us to “double up” the learning in our undergraduate courses, getting 
better value for the learner and increased effectiveness and efficiency for the institution. In this approach, 
the vast majority of courses contain a communication course outcome, key to our writing across the 
curriculum approach. All required courses also contain course outcomes in Critical Thinking, Ethics, or 
Research and Information, while elective courses contain evenly distributed course outcomes in Arts and 
Humanities, Mathematics, Science, or Social Science. Technological literacy is reinforced throughout a 
student’s program. Examples of this “matrixing” are shown in Appendix A [10]. 


As mentioned above, this method provides several other advantages including, 

• Centrally managed curriculum ensuring consistent learning objective distribution. 

• Consistent course outcomes across course sections and faculty 

• Consistent faculty training on rubric use to assure inter-rater reliability 

• Universal learning objectives with common rubrics to evaluate student learning. 


To date, we have conducted two studies on this approach. 


In the 2009 cohort study, we used three courses in sequence, with all students remaining enrolled. We 
reviewed the percentage of students achieving “practiced or higher” in the communication outcome: 
Demonstrate college-level communication through the composition of original materials in standard 
American English. Students achieving “practiced” or better increased from 76% in the first course to 85% 
in the third course, documenting steady improvement in core academic skills of students as they 
progressed through courses. 

In the 2010 ethics and communications study, the sample included 2581 BS in Psychology students 
learning at the 1-, 2-, and 400 levels. They were each assessed in ethics and communications. In ethics, 
the average scores for students were 2.72 (100 level), to 3.54 (200 level), and 3.64 (400 level) 
respectively. In communications, the average scores improved from 3.20 (100 level) to 3.49 (200 level) 
to 3.54 (400 level). Our initial conclusions are that the general education program is resulting in 
documented student improvement of core general education skills in the areas of ethics and 
communications [9]. 
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As students progress through their programs of study, their progress on achieving these outcomes is 
monitored. CL As provide feedback to students, faculty and administration about specific knowledge and 
skills acquired by a student during the course of his or her education. We use this feedback to improve the 
quality of our courses and to support our faculty’s ability to improve the proportion of students who 
achieve proficiency and mastery. 


This ability to employ technology to matrix learning outcomes within a single learning experience also 
has implications for reducing the time and cost to degree without reducing learning. If, for example, the 
outcomes embedded across the curriculum amounted to the equivalent of 30 credits, we could consider 
increasing the credit award per course and decreasing the number of courses required for graduation. By 
deepening the learning we are assessing, we further enrich the value of all learning experiences. 


C. Improving Student Support and Success through Personalized Services 

The KNEXT (Knowledge Extension) program is an online teacher-led or self-paced diagnostic program. 
Using a portfolio development approach with an independent third party evaluation of credit claimed, 
KNEXT provides a comprehensive assessment of learning gleaned from prior life experience. Enrolled 
students are actively encouraged to make use of all prior learning—degrees and certificates earned, past 
college experience, professional seminars and applicable work and life experience. 


KNEXT is a form of pedagogy as well as a powerful diagnostic for the learner. Participants learn how to 
actively and consciously reflect on their experience defining the learning that has been generated by that 
experience. 


In seeing anew what they know and how they learned it, the KNEXT process personalizes the degree 
planning and educational design process for each learner, transforming aspiration and motivation as a 
consequence. As a result, students who complete the course enjoy three significant educational benefits. 
First, they save time and money on the way to earning their degree. Data from the recent CAEL report, 
“Fueling the Race,” indicated that students who received credit through a prior learning portfolio process 
persisted to graduation within 7 years at more than twice the rate of those who did not [11]. Our data for 
KNEXT participants indicates an even more favorable success ratio. 


Second, they perform significantly better than other students in the ensuing courses that they take. As we 
track student performance inside Kaplan University, KNEXT “graduates” consistently do better in each 
course that they take subsequent to their portfolio completion. 


And third, they persist to completion at significantly higher rates. In both the CAEL report data and our 
internal data at KNEXT, students with portfolio assessment credit persist significantly longer than those 
who do not. 

Data gathered from 1062 students who enrolled in the KNEXT assessment course indicate that 81% of 
those who enrolled finished the course successfully, and more than 75% of those who enrolled in the 
course have either graduated from Kaplan University or are still enrolled [12]. 


Historically, campus-based programs like KNEXT have been hamstrung by the available communication 
and information processing systems. There has been no way to scale the programs because of the volumes 
of accumulated information in each portfolio that could only be processed by hand. Put another way, 
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logistics required that such programs be small and of a boutique nature. 


Today, thanks to the tools available through technology and social networking, assessment of prior 
sponsored and non-sponsored learning can be done to scale with rigorous third party quality assurance. 
Furthermore, the student, with her portfolio, can then “shop” her transcript at any college electronically, 
dramatically reducing the time involved in finding out which credits will transfer and which will not at 
each institution. This is a major shift towards the student in pedagogy (active reflection), access (knowing 
college standards) and responsibility (shopping for the best college fit). 

V. CONCLUSION 

What does this “new ecology” of learning mean for higher education in the 21 st century? The list of 
answers is far longer and more complex than the implications discussed in this paper. Having said that, let 
me use one more example from Kaplan University and then describe its implications. Kaplan offers its 
degree programs in four formats - on line, blended, on campus, and supported by drop-in learning 
centers. Using our outcomes-based, course level assessment program as the foundation, we can offer the 
KNEXT diagnostic service to place learners accurately on their degree paths saving them time and money 
towards their degrees. This is effective across all of the different teaching and learning models we offer. 
And we are also providing a consistent curriculum with embedded general education components across 
all of the modalities. 


So, regardless of the delivery model, the learning outcomes and rubrics used for measurement are the 
same. Importantly, the commitment to course-level assessment supported by rubrics assures consistent 
outcomes across all modalities while allowing for appropriate flexibility for the faculty member to teach 
in the manner she feels is most effective. Having a strong, clear, and consistent approach to learning 
outcomes at the course level permits more flexibility during the teaching and learning process. The 
assessment of learning becomes the distinguishing feature, the place where quality is assigned. 

We all know that when adult learners leave school without completing, often the reason is very personal 
to them and their life circumstances. Life happens, as the saying goes. Using this approach, made 
possible by technology, Kaplan University allows an adult learner to move from model to model - 
beginning on a campus, then moving to blended, online, or leaming-center-supported programs; or vice 
versa - as they need to do so. The university itself becomes a matrixed learning environment that can 
adapt effortlessly to the changing needs of its learners as their lives unfold. Without the added efficiency, 
effectiveness, and capacity that technology provides, this level of personal responsiveness coupled with 
consistent academic quality would be impossible. 


Having consistent, reliable, and valid assessments that are linked to life skills as well as workplace and 
professional abilities will become the new standard for academic quality in a world that is increasingly 
mobile with an ever-increasing need for more well-educated people. Academic quality that is derived 
from the reputation of the institution will continue to have validity in specific cases. But in an 
environment fueled by abundance, efficiency, and effectiveness where mass higher education is both 
possible and desirable, having valid, reliable, and consistent assessments of transparent learning outcomes 
will become the dominant quality standard. Having transcripts that tell the reader what the learner knows 
and is able to do as a result of her learning will be a significant quality differentiator made possible by 
technology. 


The underlying capacity that enables this change is technology’s capacity to provide not only the 
abundance, but also the efficiency and effectiveness required to support educational services with 
consistent, clear, and rigorous standards to additional millions of people. This is the key to quality- 
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assured mass higher education. 
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VIII. APPENDIX, SAMPLE OUTCOMES 


HU261: Global Civilization 


Course | Author 

E C ourse Home A 

• Syllabus 

• Flex Seminar 
Guide 

• Course Level 
Assessment 
Rubrics 

• Course Level 
Assessment FAQ 

• APA Quick 
Reference 

• KU Library 
Presentation 

• Course Questions 


Course Admin Gradebook Email Live Doc Sharing Dropbox Journal Webliography Help 


*lcome to HU261 1 Global Civiliz 



Course Outcomes 


Xj 


that define diverse cultures 

HU261-2: Evaluate leadership as a factor in 
cultural development and shifts. 

HU261-3: Demonstrate how material culture 
represents various cultural concepts through 
written communication. 



• The World of 
Humanities 

■ Twitter 

• Course Issues (For 
Faculty)* 

■ Turnltln* 

• Faculty 
Expectations* 

• Department 
SharePoint Site* 

► Unit 1 


GEL-1.1: Demonstrate college-level 
communication through the composition of 
original materials in Standard American 
English 

GEL-5.2: Analyze the impact of human 
expressions on culture 

ID: HIJ2ei-00-03-CHO 


What's New 


Figure 2. Course Outcomes for HU 261 Global Civilization 


Course | Author 

p Course Home | A 


Gen Ed Literacy Program Outcomes: 

GEL-5.2 Analyze the impact of human expressions on culture 


■ Syllabus 

• Flex Seminar 
Guide 

• Course Level 
Assessment 
Rubrics 

• Course Level 
Assessment FAQ 


Evaluation Criteria: 

This rubric is founded on the ability of students to assess the impact of humanistic achievements on culture Minimal demonstrated ability to recognize any impact of humanistic 
achievements on culture results in no skills demonstration (no progress) or the most basic understanding of the impact of human expression (introductory) The rubric moves up Bloom's 
hierarchy from superficial engagement of humanistic achievements (emergent), to clear explanation of and engagement the influences of humanistic achievements on culture (practiced), 
to clear reflection on personal connection to human expression and potential (proficient). The mastery level is reserved for student work that exceeds expectations and includes no 
errors. 

Note: Unit 7 Discussion Question 1 is the assessable deliverable for this outcome. 


• APA Quick 
Reference 

• KU Library 
Presentation 

• Course Questions 

• The World of 
Humanities 

• Twitter 

• Course Issues (For 
Faculty)* 

• Turnltln* 

• Faculty 
Expectations’ 

• Department 
SharePoint Site’ 


> Unit 1 


Not Assessed 

9 

No work was received by the instructor 

No Progress 

0 

Work demonstrates little progress satisfying the Humanities criteria 

Introductory 

1 

Work demonstrates basic familiarity with several traditions within cultures 
(criterion 1) but does not satisfy the other criteria 

Emergent 

2 

Work demonstrates familiarity with human expressions from cultures (criterion 
1) and the ability to engage well with primary source material from the 
humanities (criterion 2) 

Practiced 

3 

Work demonstrates familiarity with human expressions from cultures (criterion 

1) , solid engagement with primary source material from the humanities (criterion 

2) , and critical thinking about the past and the present in a manner that fosters 
reflection on human potential (criterion 3) 

Proficient 

4 

Work demonstrates familiarity with human expressions from cultures (criterion 

1) , solid engagement with primary source material from the humanities (criterion 

2) . critical thinking about the past and the present in a manner that fosters 
reflection on human potential (criterion 3) and uses knowledge of human 
expressions to help one assess human potential (criterion 4) or one s own 
potential (criterion 5) 

Mastery 

5 

Work satisfies all five Humanities criteria 


Figure 3. Gen Ed Literacy Outcomes Rubric for Analysis for HU 261 Global Civilization 


20 


Journal of Asynchronous Learning Networks, Volume 15: Issue 2 










































From Scarcity to Abundance: IT’s Role in Achieving Quality-Assured Mass Higher Education 


HU261: Global Civilization 


Course Admin Oradebook | Email Live Doc Sharing Dropbox Journal WebBography Help 


Kaplan Tech Support 
866-522-7747 
Option 2, then Option 0 


Course | Author 

Q Course Home | * 

■ Syllabus 

• Flex Seminar 
Guide 

■ Course Level 
Assessment 
Rubrics 

■ Course Level 
Assessment FAQ 

• APA Quick 
Reference 

• KU Library 
Presentation 

• Course Questions 

• The World of 
Humanities 

• Twitter 

• Course Issues (For 
Faculty)* 

• Turnltln* 

• Faculty 
Expectations* 

• Department 
SharePoint Site* 


> Unit 1 

00a 


Gen Ed Literacy Program Outcomes: 

GEL-1 1 Demonstrate college-level communication through the composition of original materials in Standard American English [Note Standard American English is the common 
language of educated professionals and is understood by people across different geographical regions, educational backgrounds, and ethnic and racial associations ] 

Evaluation Criteria: 

This rubric is founded on the ability of students to demonstrate college-level communication skills in the form of the composition of original writing in Standard American English. Minimal 
demonstrated ability to communicate clearly and with few errors results in no skills demonstration (no progress) or the most basic demonstration of communication skills (introductory). 
The rubric moves up Bloom's hierarchy basic, coherent communication with little organization (emergent) to the expression of a clear viewpoint and purpose (practiced), to original 
organized, communication that demonstrates a high level of logic, insight, and mechanics (proficient). The mastery level is reserved for student work that exceeds expectations and 
includes no errors. 


Note: Unit 8 Project is the assessable deliverable for this outcome. 


Not Assessed 

9 

No work was received by the instructor 

No Progress 

0 

Student work does not yet demonstrate an understanding or progress towards 
achievement of this outcome. Some content may be plagiarized 

Introductory 

1 

Student work has a viewpoint but it is not clearly stated and the work may reveal a 
lack of coherence and originality. The use of Standard American English is lacking 
There are significant problems with content organization format and mechanics 

Emergent 

2 

Student work uses Standard American English and shows coherence, order and 
some originality, but may not include a clear viewpoint and purpose Work 
synthesizes factual support, but lacks organization throughout the assignment. The 
work also includes one or two significant gaps in content does not demonstrate 
transitions, and has some problems with format and mechanics 

Practiced 

3 

Student work is in Standard American English and demonstrates the use of a clear 
viewpoint and purpose. The work is original, well ordered and coherent on the 
whole. While content is thorough and well defined there may one or two minor errors 
of omission There may also be minor errors of organization, style, and mechanics. 

Proficient 

4 

Student work includes clearly established and sustained viewpoint and 
purpose. Assignment is in Standard American English and demonstrates clear 
organization. Communication is very well ordered, logical and unified, as well as 
original and insightful. The work displays superior content, organization, style, and 
mechanics 

Mastery 

5 

Student work includes a highly developed viewpoint and purpose. Assignment is in 
Standard American English and demonstrates superior organization. Communication 
is highly ordered, logical and unified. The assignment displays exceptional content, 
organization style and mechanics 


Figure 4. Gen Ed Literacy Outcomes Rubric for Communication for HU 261 Global Civilization 
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